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ABSTRACT 

This paper describes a workshop designed to lead 
teachers toward the use of open ended response questions rather than 
multiple choice items. The rationale was based on the assumption that 
assessment will become more authentic because it will measure depth 
of understanding and deal with both content and process. Authentic 
assessment is defined as an alternative to traditional 
multiple-choice standardized achievement tests involving direct 
examination of student performance on significant tasks that would be 
relevant to life outside of school. The focus of the workshop 
described was on participants constructing questions based on a group 
of generic questions or formulas into which the teacher could insert 
his or her own content. Workshop participants were then asked to 
write answers to the questions they had prepared and to construct an 
appropriate rubric. Three handouts guiding the workshop (Using 
Generic Open-Ended Response Questions, Let's Practice Writing and 
Scoring Open Response Questions, and Samples of Generic Questions for 
Adapting to Your Own Content) are inc luded. (Author/LL) . 
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Preparing Teachers for Open Response Testing 



Ann Lockledge and Judith A. Hayn 
Abstract 

Describes a workshop designed to lead teachers toward the use of open 
ended response questions rather than multiple choice items. Rationale was based 
on the assumption that the assessment will become more authentic because it will 
measure depth of understanding and deal with both content and process. Authentic 
or alternative assessment was defined as an alternative to traditional multiple- 
choice standardized achievement tests involving direct examination of student 
performance on significant tasks which would be relevant to life outside of school. 
The focus of the workshop described was on participants constructing questions 
based on a group of "generic questions" or formulas into which the teacher could 
insert his or her own content. Workshop participants were then abked to write 
answers to the questions they had prepared and to construct an appropriate rubric. 
The handouts and directions for the workshop are included. 



Preparing Teachers for Open Response Testing 
Ann Lockledge and Judith A. Hayn 

Recently we have seen a move toward more authentic evaluation -- testing 
that involves students in answering the sorts of questions they might confront in the 
real world. Educators want to know if the student can move beyond the school 
setting and consider problems in a larger context. Classrooms teachers at all levels 
are using response journals, hands-on materials, manipulatives, projects, and 
portfolios. At the national level there is a move toward the use of open ended 
response questions rather than relying solely on multiple choice items. In some 
states such as North Carolina, standardized open-ended tests have been developed 
at each grade level for the major subject areas (Division of Accountability Services, 
1992). 

Authentic or alternative assessment (Worthen, 1993) is viewed as an 
alternative to traditional multiple-choice standardized achievement tests and 
involves direct examination of student performance on significant tasks that are 
relevant to life outside of school. Teachers have long kept anecdotal running records 
and folders of student work before such records were recognized as an legitimate 
means of direct assessment. But when the demand for accountability became 
coupled with increasing criticism of standardized multiple-choice tests, this way of 
looking at what a stude.-^t had accomplished was seen as holding considerable 
promise. It is possible that the development of standards or rubrics for judging this 
type of performance has the potential for becoming divisive. That is one of the 
reasons that the authors felt that teachers should be prepared to develop rubrics of 




their own and discus them with their students so that the children would understand 
the process. 

In assessing depth of understanding (Newman, 1992) we must be sure that 
students are asked to consider alternatives and to make choices based on 
knowledge. They need to be working with substance and critical ideas and use 
persuasion or take a theoretical stance. The problem they are asked to solve should 
be one that is likely to be addressed in the world beyond the school 

The use of open-ended questions asks students to use higher order thinking to 
organize or apply the knowledge they are acquiring. Open-ended responses differ 
from the traditional essay in that they are more pointed and require a concise answer 
that shows analysis of information presented, situations that develop, or problems to 
be confronted. This type of evaluation also requires the use of a rubric for grading. 
The rubric for a question sets out the standards a teacher could reasonably expect in 
an answer from a prepared student. It is usually based on a scale that goes from 
minimally acceptable to clearly understood and analyzed. 

The focus of this workshop was on participants constructing open-ended 
questions based on that which they in fact teach and then preparing rubrics to 
accompany each one. Only by the consistent use of such questions can students 
become comfortable with the techniques for answering and teachers with the 
techniques for grading. Even better is when students use prepared njbrics for 
scoring each others writing. The assessment will become authentic because it will 
measure depth of understanding and deal with both content and process. 

The training was based on a group of "generic questions" or formulas into which 
the teacher could insert his or her own content. The workshop participants were then 
asked to write answers to the questions they had prepared and to construct an 
appropriate njbric. Participants scored each other's answers in the same manner they 
might ask their students to try. it was suggested that another helpful technique might 



be to use one generic question, teach the students the rubric and then use the same 
formula with different content. The first and second set of answers could be examined 
for progress in understanding quality answers. 
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USING GENERIC OPEN-ENDED RESPONSE QUESTIONS 



Many open ended questions follow a pattern that may be used to fit the content of any 
grade level. Below you will find a generic question followed by possible ways to 
adapt it to each grade level. 



1. Suppose you were asked to interview 
questions that you would ask him/her. 



Grade 3 
Grade 4 
Grade 5 
Grade 6 
Grade 7 
Grade 8 



Mayor of your city 
a State Senator 
John Muir 
John Major 
Nelson Mandela 
Thomas Jefferson 



that 



List three 



has just closed a playground 
who is running for reelection 
saved the redwoods 
is British Prime Minister 
spent 27 years in jail 
helped write the Constitution 



For the grade which you teach write a response that you might hope to see. 



2. Now score your response on a scale of 0-3 using the general Rubric at the bottom of 

the page. 

3. Beiov/ is a another generic question. See how you: group would go about adapting it 

to several grade levels. 

(name people) celebrate or observe (name holiday ) . Teil why this 

holiday is important to the people that celebrate it. Tell how the people would 
feel d uring this observance. 

Grade 3 

Grade 4 

Grade 5 

Grade 6 

Grade 7 . 

Grade 8 



GENERAL RUBRIC FOR SCORING READING AND SOCIAL STUDIES RESPONSES 
0 Answer is unresponsive, unrelated or inappropriate. 

1 . Answer deals with material on a concrete, literal level that is accurate in most dimensions. 

2. Answer deals with most aspects of the question and makes correct inferences, although 
minor errors may exist. Comprehension is on an inferential level and the key skills are 
synthesis and analysis. 

3. Answer addresses most aspects of the question and uses sound reasons and cites and 
explains appropriate examples. Uses skills of evaluation as well as analysis and synthesis. 



LET'S PRACTICE WRITING AND SCORING OPEN RESPONSE QUESTIONS 



Take the questions in youi- envelope and adapt one of them co the curriculum for each of the 
grade levels. (For Social Studies: 3rd-Communities; 4th-Geography 5th-Regions of 
U.S.; 6tli Europe; 7th- Africa and Asia; 8th-United States History). Or take several 
questions and adapt them for the grade level you teach. 

After you have written the sample questions, cIiool^ one and write an answer that you would 
consider acceptable from your students. 

Exchange answers and score using the rubric at the bottom of the page or on the first page: 



SAMPLE QUESTIONS: 



WRITE AN ANSWER TO ONE QUESTION: 



SCORE SOMEONE ELSE'S RESPONSE BY USING THIS RUBRIC: 
0 No response 

1 . Responded to question and attempted an explanation. Details are sparse. 

2. Responded to question with a logical explanation. Includes specific details. 

3. Responded to questions with extended explanation. Elaborates to back up logic. 

WRITE A BETTER RUBRIC FOR A SPECIFIC QUESTION YOU WROTE: 

1- , 



SAMPLES OF GENERIC QUESTIONS 
FOR ADAPTING TO YOUR OWN CONTENT 

SOCIAL STUDIES 

Recently there have been changes in _(nanie place or process) Pick one and explain its 

effect on the lives of people. 

Make a list of ways tha't _(name event) changed the attitudes and viev/points of the people in 

_(name countiy or dme period) . 

Wars bring hardships. Tell why the (name war) was difficult for _(name people) , 

Explain one way someone might have tried to cope with these hai^dships. 

Using the map(s) of _(tell type) resources in _(name area) , circle an area where you would find 

a large city. Explain your answer. 

One problem th?^ ^identify type of person) must decide about is „(nanie problem)^. Tell what 

two different points of view might be. 

You are assigned a report to compare tlie govemment of and , Name tlaree ways 

of doing things you would need to describe or compare. 

It has been said that religious belief affects almost everything the people do. Give two 

specific examples and explain. 

All of the following statements about „(name place, person, event) ai'e true. Place a check beside 

the one you consider most important and explain your reasons for your selection. 

All places change over time. The following are examples of changes taking place in . 

Choose one and explain how it will affect the lives of the people who live there. 

If you were going to _(name a change) in (name place) , what are some of the factors you 

would have to consider? 

Name two different ethnic groups who live in . Tell how they have tried to 

maintain their traditional cultural identity. 

MATH 

The were traveling by . Using the map and the graph shown below, tell what 

a journey from to would have been like. 

SCIENCE 

In the picture of the below, there are items labeled. At least 

of them ai*e necessary for it to operate properly. Explain why ^of the items are 

needed. 

Danny has samples of , Tell two things he could do to find out which one 

of them works best . Explain why your test would give accurate information . 

READING 

After reading a book about the life of , tell why you think he chose to become 

a(n) . What things m his life made him become a great one. 



